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^/ouchers for K-12 education 
continue to attract interest, offering the 
promise of greater parental choice, 
enhanced school efficiency, and 
improved educational outcomes for 
students. The first formal voucher 
program was established in Milwaukee 
in 1990, and its practical success was 
followed by programs in Cleveland, 
Florida, Colorado, and Washington, 

D.C. With these programs has come sus- 
tained inquiry into vouchers and their 
anticipated effects. Flere we review the 
evidence that has accumulated so far 
about voucher programs in general, 
and then we take a closer look at the 
Cleveland Scholarship Program (CSP) 
in particular, including its effects on 
student test scores. 

Ohio plans on introducing a CSP-like 
program in 2006 at the lowest-perform- 
ing schools across the state. Evidence on 
the efficacy of the CSP program is there- 
fore critical, both for the direct develop- 
ment of policy in Ohio and for voucher 
initiatives in other states. 

■ Research to Date on Vouchers 

Voucher programs are expected to raise 
students’ academic achievement, and the 
most high-profile research on vouchers 
has looked at whether they do. The evi- 
dence shows, at best, moderate advan- 
tages for voucher participants. 

For the Milwaukee Program, Rouse 
(1998) found small but positive effect- 
size differences for math but no effect 
for reading. Flowever, the data were 
from the first five years of the program; 
religious schools were not participating, 
and the voucher students were concen- 
trated in a few schools. For the Florida 



program, Figlio and Rouse (2005) found 
modest results from data on over 
1 80,000 students. Voucher users in 
initially low-performing schools do post 
higher test scores, but much of this is 
attributable either to student characteris- 
tics (that is, the users were self-selected, 
high-ability students) or teaching to the 
high-stakes test (suggesting that the 
achievement gains may not have been 
genuine cognitive gains). 

Randomized field trials for vouchers in 
three cities found small test score gains 
after three years (Flowell and Peterson, 
2002). These treatment effects were 
primarily for African Americans in one 
setting, with no evidence of cumulative 
gains for those who used the voucher for 
the longest periods. Finally, it is worth 
noting that new evidence from expanded 
public school choice points to the same 
conclusion, with few achievement gains 
from placement in a choice school 
(Cullen etal, 2005). 

Other studies have investigated broader 
questions, such as: How do parents 
choose schools when vouchers are avail- 
able to them? In what ways are private 
schools accessed by voucher students 
better than public schools? How do 
vouchers influence public finances? 

In looking at school-choice decisions by 
parents, it is clear that many affluent 
families already have choices; attention 
therefore focuses on how voucher pro- 
grams might open up choices for those 
families that are constrained. Thus far, 
all voucher programs have been targeted 
to low-income families or to districts 
with low-performing schools. Clearly, 
vouchers expand the choices of low- 
income families. 




Voucher programs are intended to raise 
the academic achievement of students, 
hut, unfortunately, so far the evidence 
suggests that Cleveland’s voucher stu- 
dents perform no better than their 



counterparts in public schools. 




However, several mediating factors mean 
that voucher programs are less effective 
in promoting choice among low-income 
families than is implicit in a simple read- 
ing of the program eligibility rules. First, 
religion pervades family preferences 
of schools (Campbell et al, 2005). Cer- 
tainly, parents value high test scores, but 
preferences are varied, and many fami- 
lies choose their neighborhood school. 

Second, race has a strong influence on 
people’s choice of school. The relation- 
ship is complicated by different patterns 
across Hispanic and African American 
children and by the fact that public 
schools show strong patterns of racial 
segregation. However, vouchers consis- 
tently lead to greater student segregation 
(even in a highly minority public school 
system such as Washington, D.C.). 

Third, school choice is an action — it 
requires that parents actually make use 
of the voucher to change schools. Usage 
rates are much lower — perhaps by one- 
third — than offer rates (Howell and 
Peterson, 2002, table 2-2). Moreover, 
even within low-income groups, the 
children most likely to use the voucher 
are those who are the most motivated. 
Also, a nontrivial proportion of those 
who receive vouchers are already in pri- 
vate school. Thus, it is an overstatement 




to declare that vouchers significantly 
help the disadvantaged. 

Research has also investigated the 
supply of private schooling. Without 
supply, family preferences become 
meaningless — and if private schools 
do not operate in ways distinct from 
public schools, there will be no advan- 
tage to students from choosing them. 

Again, several consistent findings 
emerge. Most participating schools are 
religious (across various faiths); secular 
schools have a small market share. The 
supply of new schools appears reason- 
ably elastic: For example, almost 
one-half of schools participating in the 
Milwaukee program were founded after 
it was introduced. And voucher-student 
enrollees are increasingly a majority 
within their school: by 2001, 40 percent 
of the schools participating in the 
Milwaukee program had more than 
80 percent of their students claiming 
vouchers. So voucher programs attract 
new private schools rather than extra 
places in existing private schools. 

Flowever, research on the inputs and 
technologies that private schools use 
(beyond selecting their students) and 
which are more efficient has yielded 
very little: Economists are still no fur- 
ther ahead in identifying the separate 
benefits of ownership, innovation, and 
technical efficiency; that is, they are no 
further ahead in knowing which inputs 
work best (Flanushek, 2004). After con- 
trolling for student characteristics, most 
research finds only very modest advan- 
tages to private school. 

Finally, greater competition is likely to 
improve schools’ performance (Belfield 
and Levin, 2002), but the competitive 
pressures exerted by small-scale voucher 
programs are likely to be very modest. 

Other research has examined how 
voucher programs might impinge on the 
existing public school system and its 
local financing. Because schooling is 
financed locally, individual school 
choices will feed back into house prices, 
district spending, and so school quality. 

Thomas Nechyba found several novel 
results in a study he conducted in 2003, 
in which he modeled the introduction 
of large-scale voucher programs and 
examined the effects on public school- 
ing and finance. First, competition for 
high-ability students would increase; 
these students would pay lower tuitions. 



both as a result of the voucher and as a 
consequence of schools’ greater eager- 
ness to enroll them. Second, public 
schools would engage in more ability 
tracking to prevent high-ability students 
from switching to private school. Both 
factors suggest further educational 
inequalities, with greater rewards (and 
resources) for high-ability students. 
Third, public school quality is most 
likely to decline in middle or high 
income school districts. 

Importantly, opinions about vouchers are 
likely to be driven by perceived effects 
on property values than on educational 
outcomes; homeowners may be wary 
of education reforms that may raise 
uncertainty as to the value of their home. 

■ The Cleveland Scholarship 
Program 

The Cleveland Scholarship Program 
operates in the Cleveland Municipal 
Schools District (CMSD), which has 
75,000 students across 130 schools 
(2005-2006). Eligible schools are non- 
public chartered schools located within 
the CMSD and approved by the state 
superintendent. Surrounding public 
school districts are eligible to apply. 
Currently 5,734 students participate. 

Initially, eligible children had to be 
entering a grade between kindergarten 
and eighth grade, reside within the 
CMSD, and require no segregated spe- 
cial education. Low-income families 
were given preference, with those below 
200 percent of the poverty level provided 
with 90 percent of tuition or $2,250, 
whichever was lower. Families above 
200 percent of the poverty level were 
provided with 75 percent of tuition or 
$1,875, whichever was lower. About 
one-quarter of students came from the 
latter group. In the first year of operation 
(1996), enrollment was 1,996, with total 
funding of $5 million (with transport 
paid by the CMSD). 

In comparison, per-pupil expenditure 
in the CMSD in 1996 was $6,675 
(excluding transport). In 2003-04, the 
scholarships were made available for 
ninth grade and beyond. Funding was 
increased to $3,000 for grades kinder- 
garten through eight and set at $2,700 
for higher grades. Where voucher appli- 
cations exceeded available placements, a 
lottery system was used. 

Despite its usefulness for infoming 
future voucher reforms, the CSP has 
been the subject of little academic 



inquiry. The program is sufficiently 
large to allow for samples of students 
according to voucher status and reli- 
gious schooling, and with recent data, 
it offers an up-to-date evaluation of 
vouchers in light of recent school-choice 
reforms. Given the duration of the 
program, it is possible to examine the 
question of whether persistence in the 
program yields higher rewards. Impor- 
tantly, the CSP voucher is relatively 
ungenerous: If effects can be found for 
this program, it is likely that achieve- 
ment gains would be even larger for 
programs which are more generous. 

Students were classified according to 
voucher status so that effects could be 
identified: Users (offered a voucher and 
used it to attend private school); non- 
users (offered a voucher but did not use 
it or stopped using it); applicant rejects 
(applied for a voucher but not offered 
one); eligibles (eligible to receive a 
voucher but did not apply for one); and 
ineligibles (did not apply and were not 
eligible according to the program rules). 

When studying the achievement data in 
places where voucher programs are in 
place, various biases could make identi- 
fying the effects that are due just to 
vouchers more difficult. Three biases are 
particularly important, but their effects 
are (probably) offsetting for this program. 
Applicant bias occurs where only those 
who apply for the voucher are likely 
to benefit from it. This will bias gains 
toward users, because applicants are 
typically motivated families. Eligibility 
bias occurs where those who are eligible 
differ both from those who apply and 
(separately) from those who do not 
apply. This will bias gains away from 
users, because CSP eligibility is condi- 
tional on low family income. (It is possi- 
ble with these data to control for eligi- 
bility). Usage bias occurs as those who 
use the voucher differ from those who 
do not use it, conditional on application 
and eligibility. This will bias gains in 
favor of users relative to nonusers 
because usage is positively correlated 
with ability and family resources. In 
addition, data collection inevitably 
generates some response bias as survey 
attrition rates are higher for nonusers. 

■ Evaluating the CSP 

Since 1996, the CSP has been evaluated 
by the Center for Education Evaluation at 
Indiana University (see Metcalf et ah, 
2003). The center began collecting data 
for students entering kindergarten in 



TABLE 1: EFFECT-SIZE TEST SCORE GAINS OVER PUBLIC SCHOOL STUDENTS 



Reading Math Language 

Second grade 



Voucher user 


-0.060 


-0.092** 


0.097** 


Nonuser 


-0.019 


-0.021 


0.162 * 


Rejected applicant 


0.083 


0.026 


0.136 


Number of students 


1,733 


1,786 


1,736 


Fourth grade 








Voucher user 


0.042 


-0.113*** 


0.038 


Nonuser 


-0.065 


0.044 


0.076 


Rejected applicant 


-0.030 


-0.055 


-0.063 


Number of students 


2,089 


2,102 


2,085 


NOTES: 1 . Terra Nova test. OLS estimation. Effect sizes control for: subject-specific first^third grade 
scores; African American; Hispanic; female; free lunch (a proxy for income); unsubsidized lunch; 
class size; years of teacher experience. 2. *significant at 10 percent; **significant at 5 percent; 
***significant at 1 percent. 




1997 and has continued until they were 
in the sixth grade in 2003. (No substan- 
tive changes in program design occurred 
during this period, but the legal status of 
the program was resolved only in 2002). 
The dataset includes over 4,000 students 
who attend over 100 separate schools. 

In terms of voucher status, the dataset is 
composed of users (23 percent), 
nonusers ( 1 0 percent), applicant rejects 
(16 percent), and a public school com- 
parison group (51 percent). 

The dataset has three advantages: It is 
longitudinal (including achievement 
measures); it includes students from 
multiple comparison groups; and it is 
a large sample spread across many 
different schools. However, because the 
data do not allow us to distinguish 
eligibles from ineligibles, both of these 
are included in the public school 
comparison group. Also, the sample 
sizes are considerably lower with 
attrition and missing responses. 

Evidence on the CSP comports with 
extant research. Specifically, most stu- 
dents chose religious schools, and high 
(and growing) proportions of voucher 
applicants and users had previously 
been enrolled in private school. Student 
characteristics across voucher status are 
similar to those in other studies. African 
American students and low-income 
students are much less likely to use the 
voucher, conditional on being offered it. 

To identify the achievement gains from 
the voucher program, we estimate a 
series of education production functions. 
Test scores in second and fourth grade 
are regressed against a set of student and 
school characteristics, including prior- 
year test scores. The Terra Nova test is 
applied: widely used across U.S. 



schools, it measures basic skills across a 
range of subjects. It is norm-referenced 
to allow comparison between students. 
The impacts of voucher status on 
achievement are reported in table 1 . 
These impacts are effect-size gains rela- 
tive to the public school comparison 
group in reading, math, and language. 
(“Effect sizes” are a statistical method 
used to assess the magnitude of an effect. 
In this and similar studies, an effect size 
is the gain in achievement divided by the 
standai'd deviation of achievement; a 
gain of 0. 1 is therefore one-tenth of a 
standard deviation). 

The top panel of table 1 shows mixed 
effects according to voucher status. For 
reading, there are no differences across 
the four groups in second grade. For 
math, voucher users report the lowest 
scores. In contrast, for language, the 
public school group does considerably 
worse than the other three groups; how- 
ever, voucher users gain the least — ^both 
nonusers and rejected applicants show 
larger advantages. Given the biases that 
might lead to statistically significant 
gains for voucher users, we carmot find 
evidence that voucher students outper- 
form relevant compaiison groups in 
second grade. Moreover, the bottom 
panel shows that by fourth grade the 
math penalty for voucher users persists, 
but the gains in language have dissipated 
to insignificance. 

Additional testing using second grade 
data affirms the weakness of any 
voucher effects. When we do not control 
for prior achievement, the results do not 
favor voucher users: The math penalty is 
maintained, and the language advantage 
over the public school group is elimi- 
nated. Moreover, the rejected applicants 
report statistically significant test score 



gains in reading and language. When 
we compare users only against those 
who were not offered a voucher (the 
treatment effect), users report scores 
that are lower by 0. 14 standard devia- 
tion in reading and 0. 1 1 standard devia- 
tion in math, with no difference in lan- 
guage. When we examine whether 
those who have participated the longest 
in the program obtain the strongest 
effects, we find mixed effects across the 
three subjects. 

Finally, we test for whether the impact 
of vouchers differs by race. Restricting 
the sample to black students, voucher 
users appear more disadvantaged: Their 
reading scores are now statistically sig- 
nificant and lower; the math penalty is 
now not statistically significant, but 
remains negative; and the language 
advantage is not discernible. These 
subanalyses give no indication that 
vouchers have a differential and benefi- 
cial impact for black students. 

Overall, there is no clear advantage for 
voucher students; if anything, there is 
a slight academic penalty. However, 
given that in 2001 the value of the 
voucher was less than $2,400, and the 
opportunity cost in the public school 
system was approximately $8,800, it 
might be concluded that the voucher 
program is cost-effective. Yet, back-of- 
an-envelope calculations show that this 
conclusion is premature. The CSP 
voucher does not include transporta- 
tion, which must be paid by the district 
and which may be as high as $ 1 ,400 
under a large-scale voucher program 
with children going to many different 
schools (see Levin and Driver, 1997), 
and standardized assessments, which 
public schools must impose. It does not 
serve special education students or the 
most disadvantaged students (who do 
not make use of the voucher). And 
because the program is subadditive, for 
every three vouchers distributed 
approximately one student would have 
attended private school an 5 way. There 
are also additional administrative costs. 
Finally, the $8,800 figure for CMSD is 
the average cost per student, not the 
marginal cost. With approximate costs 
of transportation, assessment, special 
education, subadditivity, and adminis- 
tration taken into account, the antici- 
pated fiscal savings falls from $6,400 
($8,800 - $2,400) to perhaps $2,500- 
$3,000. Although this is still a sizeable 
cost savings, it does not account for 
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marginal costs, student disadvantage, or 
any reorganization costs. 

■ Conclusion 

Recent research sheds light on the effi- 
cacy of vouchers. Broadly, it may be 
questioned whether vouchers — even 
with some modest achievement gains — 
will be a catalyst for educational 
improvements. On the demand side, 
there are many other factors influencing 
the choices parents make. On the supply 
side, most participating schools are reli- 
gious, with little evidence of new secu- 
lar schools opening or accepting vouch- 
ers. Also, there is limited guidance on 
what makes for an efficient private 
school. Thus, competition between 
schools will probably be muted, and 
nonreligious students are unlikely to 
have more options. Finally, general 
equilibrium models explain why home- 
owning voters are wary about expanding 
voucher programs. 

The Cleveland Scholarship program 
readily fits into this general pattern, 
evincing similar features to voucher 
programs in Milwaukee, Florida, and 
Washington, D.C. Although targeted at 
students from low-income families, 
these programs in fact serve those some- 



what closer to the middle of the income 
distribution when usage rates and prior 
schooling are accounted for. Students 
are highly likely to be in religious 
schools, and black students are less 
likely to use their voucher. 

Importantly, the CSP results are not 
encouraging with respect to achieve- 
ment: The program shows a slight acad- 
emic penalty for voucher users relative 
to other comparison groups. Thus, it 
seems unlikely that an expanded state- 
wide program will radically enhance 
educational outcomes in Ohio. 
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